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Abstract- Sign languages use visual pattern to communicate 
rather than acoustic patterns that are communication mode in 
verbal communication. Sign languages being the most 
structured form of gestures can be considered as the benchmark 
for systems of gesture recognition. SLR has got its applicability 
in the areas of appliances control, robot control, interactive 
learning, industrial machine control, virtual reality, games, 
simulations etc. apart from its significance for hearing impaired 
community. The paper aims to present a systematic, robust, 
reliable, and consistent system for static Pakistani Sign 
Language (PSL) recognition. The paper is based on empirical 
evaluation of different classification techniques for SLR. This 
pragmatic approach leads to a Fuzzy Model (FM) that has 
shown very high accuracy rate for PSL recognition. Sign 
languages have inherent uncertainty, so SLR systems demand a 
classification method that give due consideration to this aspect 
of uncertainty. This is the reason for selecting fuzzy inference 
for the proposed SLR system and experimental analysis has 
proven its suitability for SLR. The meticulous statistical analysis 
performed for proposed PSL-FM has shown very promising 
results. 

Keywords-pattern recognition;fuzzy classifier; sign language; 
classification. 

I. Introduction 

The increasing technological developments and 
advancements in the world of machines are leading to more 
and more demanding users. Human beings want machines to 
interact with them in more natural way. The users of the 
machine wish to have an interaction with machine in the same 
pattern as they have with their fellow beings. With every 
progressing day, machines are becoming part of human life, 
more and more. This intermingling of machines and human is 
asking for removing the barriers in the communication 
channels between human and machines and hence researchers 
are having their focus to make this channel as natural as 
possible. Human needs speech and gestures to communicate 
and same is the communication requirements from machines. 
So these human like modes of communication should be 
equipped into the machines. The domain of speech as mode of 
communication is well researched but gestures as mode of 
human-machine communication still have a lot of room for 


research. Another motivation behind research in the domain of 
gesture recognition is the opening of new aspects of 
communication for the hearing impaired people. For deaf 
community, Sign Language (SL) as a kind of gestures is the 
primary communication media. Sign language is considered to 
be the most structured form of gestures. There are millions of 
deaf people around the world, who are using sign language for 
accessing and exchanging information. This is the reason that 
in recent years, SLR is in focus and researchers came up with 
variety of solutions. Sign language is the language of visual 
gestures that are mainly used as a communication tool for deaf 
community. Sign languages use visual pattern that are used to 
communicate rather than acoustic patterns that are used in 
verbal communication. Being most structured form of 
gestures, Sign language can be taken as benchmark for gesture 
recognition system. There are many communities in the world, 
because of having high number of deaf people; the whole 
community is using gestures instead of verbal communication. 
Some examples of such communities are Martha's Vineyard 
Sign Language (USA), Kata Kolok (Bali), Adamorobe Sign 
Language (Ghana) and Yucatec Maya sign language (Mexico) 
[ 15 ]. 

Sign languages, just like acoustic languages, have got vast 
variety. SLs have got diversity in the base language and then 
these base languages have variety in dialects. Different 
countries have different signlanguages and then there exist 
difference of dialects in different regions of same country. 
Generally, automated SL recognition (SLR) is performed in 
following phases: image acquisition, gesture segmentation, 
feature acquisition and classification. 

Signer’s image is provided as input to Sign Language 
Recognition system. This input image is then passed onto the 
segmentation module that segments out the region of interest 
i.e. hands, face, arm or may be foot, depending upon the scope 
of the recognition system. This segmented image is then goes 
for the feature acquisition module. This module extracts 
required features for the classification. Classification module 
receives the extracted features andapplies some classification 
algorithm to give the results of the recognized sign as output. 

SL is not internationally uniform; rather it varies from 
country to county and region to region. Researchers have 
proposed different SLR systems for different SLs. Some 
languages are more in focus by researchers such as Chinese 
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Sign Language CSL [22,23,24], American Sign Language 
ASL [19,20,1,2] and Arabic Sign Language 
ArSL[25,26,27,28]. Unfortunately, Pakistani Sign Language 
(PSL) could not get its due share in this SLR development, 
kausar and et. al. [8] has proposed a fuzzy classifier using 
colored gloves for PSL alphabets recognition. Khalid and et. 
Al [16] has made use of data gloves for the recognition of 
static one handed PSL alphabets. There are approximately 
nine million people (5% of total Pakistani population) in 
Pakistan with hearing impairments [14] and out of these 1.5 
million are deaf [14]. Pakistani Sign Language (PSL) is a 
visual-gestural language, it is a blend of Urdu(national 
language of Pakistan), and some regional languages. Less than 
ten thousand Pakistani deaf have attended school [14] because 
the information exchange methods prevailing in the schools 
are not sufficiently fulfilling their needs. Bringing PSL in 
modern technologies using SLR can improve the lives of deaf 
community in many aspects. SLR is a complex domain and 
has many inherent challenges with respect to region of interest 
segmentation, signer’s environment, descriptor’s invariance, 
sign articulation, variation in sign dictionary, selection and 
extraction of representative sign descriptors, correct, efficient 
and reliable classification. 

The paper presents a reliable and efficient method for 
PSLR. The proposed method uses static one-handed signs. 
Classification method is the main contribution of the paper, 

II. Related Work 

The domain of sign language recognition can be used as 
benchmark in many other gesture recognition systems. This 
diverse applicability of the domain has compelled the focus of 
many researchers towards it. This focus has led to the immense 
and valuable contribution. 

Multiple cameras have been used by Elons et al. [5] and 
Dreuw et al. [1] for recognition of SL. Two different viewing 
angles have been utilized by Elons et al. to generate image 
features using pulse coupled neural network for recognition of 
American Sign Language. Special angle setting is required for 
mounting cameras to get 3 dimensional features. So system 
requires multiple cameras and complex angle adjustments for 
accurate recognition. Priyal et al. have utilized Krawtchouk 
moment features for recognition of static hand sign [9] with a 
claim of signer independence and incorporation of camera 
angle variation. Li. et al. have recognized Chinese sign 
language using portable accelerometer and surface 
electromyography sensors [3]. High accuracy rates are 
claimed using hand shape, movement and orientation. Data 
gloves have been used byShijian et al. [10] and Tan et al. [11] 
for SLR. Data gloves raise the accuracy rate but system’s 
dependence on external gadgetry is not appreciated and 
accepted widely. Colored and marked gloves have been used 
by Akmeliawati [12] and kausar et al. [13 for SLR. Prime 
points of hand are marked with special colors; this marking 
has made the recognition easier. The size of dictionary of signs 
for SLR systems is quite small in many cases. Al-Alali et al. 
[6] proposed a system for only four signs. Wassnerr et al. 
developed a system that was for recognition of only two 
French signs. Zafrullah et al. [2] have proposed American Sign 


Language recognition system to recognize only six signs. 
Ershaed et al. [6], Brashear et al. [2], Xing et al. [4] and Helen 
et al.[13] have recognized SL by utilizing depth cameras. The 
depth cameras are quite costly. 

This section has presented few contributions in SLR 
domain. It is observed that different proposed methods have 
their own strengths and weaknesses. The proposed method has 
undertaken all these weaknesses of SLR systems into 
consideration and tried to reduce these significantly in the 
proposed system. 

III. Methodology 

The paper presents PSL recognition system using a novel 
method. Signer’s image is provided as input to the system. 
This input image then processed by the segmentation module. 
The segmentation module segments the signer’s hand. The 
segmented hand is then transformed into one dimensional 
signature. This signature is then parameterized to obtain 
polynomial coefficients. These polynomial coefficients are 
then fed into the fuzzy classifier as feature vector. Classifier 
then classifies the sign and displays the output. Rest of the 
paper is providing the details of these modules. 

System initiates with the input image of signer. The image 
frame only has one hand. Signer has to wear full sleeves shirt. 
Signer’s background should be non-occluded. These 
constraints make the segmentation process, simple and 
accurate. The system does not require expensive and 
cumbersome external gadgetry for input; rather it purely relies 
on visual input. Ordinary 2-D camera is used for image 
capture; range cameras have associated barriers of 
computational complexity, cost, and special experimental 
environment. The proposed image capturing makes the 
system, cost effective and more adoptable. The captured signer 
image is then passed to the segmentation module. K-means 
clustering is used for the segmentation of hand. K-means 
clustering is used to segment out skin color pixels from the 
background. To improve the segmentation, morphological 
operations are performed on the segmented hand. The output 
of the segmentation module is the binary image of the signer’s 
hand. This binary hand image is then further processed for 
extraction of sign descriptor. 

A. Sign Descriptors 

After the conversion to binary image, the image is 
processed further for extraction of sign descriptors. Signature 
based features are used as descriptors. The descriptor used, is 
very robust and efficient. The descriptor set is quite compact 
but results are very promising. The binary image of hand is 
transformed into one dimensional shape signature. These 
signatures are then used to approximate the parameters of the 
signatures. Different types of parameterizations are 
experimented for PSL recognition. Gaussian, Polynomial, 
exponential and sine parameterization is applied on the shape 
signature. Different numbers of coefficients for each 
parametric class are tried out to check their appropriateness for 
SLR. These coefficients are then used as sign descriptor for 
classification. Polynomial parameterization gives the best 
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results[17]. The process of PSL sign descriptor extraction can 
be summarized with the following figure. 



Figure 1: Sign Descriptor Extraction 


i. Sign Contour Extraction 

For transforming sign to its corresponding 1-D signature, 
centroid distance is a very popular method. For obtaining 
centroid, segmented binary image 5(x, y) is obtained as a 

result of segmentation. Let f is the original image having hand 
sign, where ¥ (x, y) is the point at (x,y) location in the original 

image. Transformation of ¥(xi, yi) ->5(x i; yf) in such a way 
that ¥(xi, yi) E ^ would be assigned value 1 in the transformed 

image 5 and rest treated as background. So binary segmented 

image J is: 

5(x,y) = 1 iff(x,y) e c[ 

5(x,y) = 0 otherwise 

Where, ‘c[’ is the domain of binary segmented image of 
sign. Centroid (C(x), C(y)) of the hand sign is the center of 

mass Vbi E c[. whereby 5(x i; yf) . It can be obtained as 
follows: 


N 

CM = 

i=l 

N 

C(y) = 

i= 1 

Where N, is the total number of points in the segmented sign 

i.e. N= 'Zfij E If n cj .So only point v t * to be considered 

is, that holds following property: 

( i. * * * * * * * * x i>yd\ C(. x i>yd = 1 


After obtaining (Q(x), C(y)X 1 -Dimensional signature X t 
needs to be extracted. Hand sign contour points are required 
for this. Contour points are extracted by taking E (jf n 
0 E ~d[. Where R(#i) is the neighbor of v t . 



Figure 2: plot of centroid poin(red)t and contour points(green) 


ii. One dimensional sign Signature 

Once the contour points of hand are calculated, A { is 
obtained using centroid distance Let the contour points of the 
hand sign are represented with a vector 
P = 

[(^ O). Pi (y)) - (p 2 (x), p 2 0)) .(p n (x), p n (y))] ■ h 

can be obtained as: A t = |p.(x) — C(x), p^(y) — C(y) I 
is the i th distance point betweeni* contour point and 
centroid of hand sign. The centroid point of the hand sign is 
CM and C(y) and the i th contour point of hand is p.. 

For time efficiency, sampling of contour points is performed 
before extracting the shape signature. To incorporate the size 
variability of hands, adaptive step size for taking samples 
from contours is used. 

Centroid distance based signature is translation, rotation and 
scaling invariant. 



tr 


V\a^~aA/ 


Figure 3: 1 -Dimensional signatures for signs of PSL 


B. PSL Sign Model 

Mathematical modeling and parameterization is very new 
concept in the area of SLR. Polynomial modeling of PSL has 
been empirically proven best among other competitive models 
[17]. Fitting a polynomial mathematical function of a 
particular degree to the sign signature is termed here as 
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Figure 4: uncertainty in signs 

polynomial parameterization. A single polynomial function is 
fitted to each sign signature while capturing the overall trend 
of the points of sign signature. 

Sign signature is the interpolation of the data directly 
acquired from the sign. This interpolation gives one 
dimensional description of the data and this description can be 
directly used as feature vector say by transforming it into 
Fourier domain. But this transformation overhead and then 
such a big feature vector can no way lead to an efficient and 
cost effective method for SLR. PSL sign model deals with 
these issues. Instead of interpolation, the method goes for 
fitting a general function cb(x) approach. The fitting of 
function cb(x) involves, empirical evaluation of number of 
coefficients for a function class, tuning these coefficients to 
get the ‘best’ representation of the sign signature. ‘Best’ 
representation is evaluated using approach of least square 
residual. Let f be a vector having size=n, J[ = 

U i; f 2 , f 3 , f n ], and let cb be a vector of size n such that cb 

= [(b^Xi), cb(x 2 ), cb(x 3 ) , cb(x n )]. Error minimization can 

be defined as: 


n 



# = cr,-< p(Xi )) 2 

The total number of data points in the sig nature is n, i th 
actual contour point is f. and the value approximated through 
mathematical model is .cb( x i). 

d)(Xi) with minimum value for s function is selected. 
Polynomial function can be defined as: 

n 

cpUi) = 9 0 + 9j x ! 

7=1 

Order of polynomial function is denoted with n, for the 
proposed function, n goes from l->9. ith coefficient of the 
polynomial function is represented with gi. So a 
polynomialfunction of degree nine would actually return 10 


analyzed in analysis section. 

coefficients i.e. 0->9. We need to get such values of g that 
give minimum 

value for s for cb(Xi). Experimentation is also performed to 
explore the appropriate numbers of coefficient. The 
polynomial mathematical model for PSL is static, explicit, 
discrete, deterministic and inductive. 

C. Classification 

Soft classifiers are suitable for the problem domains, 
where working with crisp boundaries for classification is not 
very effective. The inherent uncertainty in certain problem 
domain pursues to opt for a classifier that is having a soft 
decision capability instead of the hard one. The soft classifier 
respects the uncertainty and provides a flexibility in the 
decision making process. 

i. Uncertainty 

The special consideration for uncertainty is required for 
SLR. The sign itself have a lot of uncertainty. Human mind is 
capable of dealing very well with the uncertainty. Human 
mind does not work on crisp boundaries for recognizing sign 
i.e. it does not need to define the accurate angles of finger or 
precise level of tilt in the fingers; rather human brain works 
well with qualitative term. For instance two signs of PSL 
naming “do” and “yay” are different just on the basis of the 
position of thumb and finger alignment, if we try to find the 
exact boundary between these two signs it would be difficult. 
Another such example is “bay” and “char”. 

The same sign posed by different signers would definitely 
possess some variations and if the signs are strictly defined in 
the terms of their Examples of straightness or tiltness of the 
fingers then their slightly rotated variants would be difficult to 
get accommodated within the hard and crisp boundaries of 
conventional statistical tools. 

Expectations from the conventional classification tools to 
deal with such uncertainty normally end up with 
disappointment. As traditional statistical classification tools 
does not respect the importance of the uncertainty and these 
tools lack the capability of dealing the qualitative term in a 
desired manner. These tools tend to transform the qualitative 
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terms to the rigid and hard quantitative terms. These are the 
reasons to look for some unconventional classification tool 
that incorporates the inherent uncertainty of sign language. So 
fuzzy classification can be very suitable choice in this regard. 
This claim for suitability is proved and analyzed in analysis 
section. 

ii. Fuzzy Classification 

Fuzzy classification is based on fuzzy inference process. 
Fuzzy inference process is basically mapping of input to 
output, and this mapping is materialized by fuzzy logic. There 
are three main steps in fuzzy inference process i.e. 
fuzzification of inputs, applying inference rules and then 
defuzzification of output. 



Figure 5 Fuzzy process 


The first step for fuzzy inference process is to fuzzify the 
input variables. Input is always a crisp value within a universe 
of discourse and that input is fuzzified by utilizing the concept 
of membership function. A curve that is mapping every point 
from input space to corresponding membership value between 
0 and 1, is called membership function. The output of this step 
is a fuzzy value that is transformed from the crisp input into 
the degree of membership for all the qualifying membership 
functions required by the fuzzy rules. This process is repeated 
for all the inputs. The degree of membership always ranges 
from 0 -> 1. For the paper Gaussian membership function is 
used for the input and triangular function for output 
membership function. 


mt26 *21 m*fl6*&»*J2 tGMHK *2*35 nMH2 *31 



(a) 

n*1 mfW "THE *3 *20»Ct *0 iflttMIrfllMO *3*2*01 4 



mfl aC ml3 inH mfM iff m(8 irenfl Onfl Wlifli1*i1»il2«C1ii(»WiiQm(Z3ri30iiW1iil33nl3)il3M3a^^ 



‘ (c) 

Figure 6 : membership functions (a) §_0(b) g_l(c)output 

The fuzzy logic is comprised of if-then statements. These 
if-then statements used for fuzzy inference are called fuzzy 
rules. These rules are applied once the input values are 
fuzzified. Some examples of if- then rules can be as follows. 

There are three sub-steps of applying fuzzy rules; 
Applying fuzzy operator, Applying implication method and 
aggregation of outputs. If the antecedent part of the fuzzy rule 
has multiple terms then there is a need to apply fuzzy operator. 
The value of antecedent obtained after applying fuzzy 
operator, is then used by the implication method to get the 
shape of the consequent of each fuzzy rule. Consequent is a 
fuzzy set that is shaped by a corresponding membership 
function. The output of the implication process is the reshaped 
consequent using the consequent’s membership function 
based on a single number. 
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After applying implication method for reshaping the 
consequent membership function for each rule, the 
aggregation of the output needs to be performed. The 
aggregation is the process of combining the fuzzy sets of all 
the rules in a way to give a single fuzzy set to make the 
decision. The input to the defuzzification step is the aggregate 
fuzzy set. This fuzzy set cannot be directly evaluated to make a 
decision as it possess a range of values. So defuzzifier resolves 
the fuzzy set up to single number. There are many methods 
that can be applied for defuzzification on the aggregate fuzzy 
set. This number is the output of the complete fuzzy inference 
process. 

IV. Results 

Standard data set is not available for PSL. So 
self-developed data is being used for this paper. Dataset is 
named as Dataset PSL Table 1. 


If (P Q is in rangel and p x is in rangel and p 2 is in 
rangel and p 3 is in rangel and p 4 is in rangel and p 5 is 
in rangel and p 6 is in rangel and p y is in rangel and 
p 8 is in rangel and p g is in rangel) 

Thenaliph 

If (P Q is in range2 and g is in range2 and p 2 is in 
range2 and p 3 is in range2 and p 4 is in range2 and p 5 is 
in range2 and g 6 is in range2 and p 7 is in range2 and 
g 8 is in range2 and g 9 is in range2) 

Then bay 

If (P Q is in range3 and p ; is in range3 and g 2 is in 
range3 and g 3 is in range3 and g 4 is in range3 and g 5 is 
in range3 and p^is in range3 and p 7 is in range3 and 
g 8 is in range3 and g 9 is in range3) 

Then pay 


Table 1: Datasetpsl 


Dataset 

Number 
of Signs 

Number 
of Signers 

test 

set 

training 

set 

DatasetpsL 

44 

10 

279 

387 


PSL recognition is quite challenging, as it has many 
visually similar signs. There are many examples in Dataset 
PSL, where same sign having different orientation is 
representing different sign. e.g. “daal” and “ain”, “zaal” and 
“ghain” and “zaal” are such examples. Similarly there are 
other examples in PSL, where signs are visually quite similar 
to each other. Such as” tuay” and “zuay”, “dal’ and “tuay”, 
“aliph” and “tay”, “ain” and “ghain” etc (Figure. 8). 
Complexity of problem is added due to the aspect of inherent 
uncertainty of SL. These challenges are well incorporated in 
proposed method. 



The proposed method named as PSL Fuzzy Model 
(PSL-FM), is compared with other alternative models. The 
following table is showing the comparison of the proposed 
classifier (PSL-FC) with other classifiers while using the 


proposed sign model as feature set for all of the classifiers. 
Support Vector Machine is a potential candidate to get 
comparison with but it is a binary classifier and 44-class 
problem compel to have a hierarchal implementation of SVM. 
44 level hierarchal model of SVM needs too many 
computational and time resources. This is why SVM is not 
giving high accuracy as expected for PSL. One-versus-all 
method is followed here for multi-class SVM implementation. 
So for a 44-class problem, the SVM considerably loses its 
strength of classification. 

Table 2: Comparison of proposed classifier (PSL-FC) with other 

CLASSIFIERS 


Classifier 

f 

Accuracy 

3-NN 

0.69 

74.8% 

5-NN 

0.63 

63.4% 

7-NN 

0.58 

58.06% 

SVM 

0.64 

68.1% 

Naive Bayes 

0.35 

35.4% 

PSL-FM 

0.98 

98.2% 


True positive, false negative and false positive rate is 
considered to be a very effective gauge for the performance of 
any classification system. 
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Table3 Comparative analysis of Different PSL 

RECOGNITION SYSTEMS WITH PROPOSED MODEL 


Proposed By 

Methodology 

No. of 
Signs 

Accuracy 

% 

Yousaf et.al [] 

Neural Network 
using data gloves 

24 

84% 

Kausar et. al 
[8] 

Fuzzy Classifier 
using geometric 
features extracted 
from color coded 
gloves 

35 

92.1% 

Alvi et al. [7] 

Statistical 
Template 
Matching using 
data gloves 

26 

85% 

Alvi et al. [7] 

Statistical 
Template 
Matching using 
data gloves 

33 

69.1% 

PSL-FM 

(Proposed) 

Fuzzy Classifier 

44 

98.2% 



Recall and precision are established statistical analysis 
measure. These are defined in terms of true positive and false 
positive. Recall and precision defines F measure. F measure 
is used to analyze the PSL-FM. 

F= 2* (Precision * recall) / (Precision + recall ) 

Precision = true positive/ (true positive + false positive ) 

Recall= true positive/ (true positive + false negative ) 


about the high accuracy achievement of the proposed 
system. The proposed system is developed for bigger dataset 
as compared to the other PSL recognition systems;even then 
its accuracy rate is much higher than others. 

There is very small part of research contribution towards 
the Pakistani sign language recognition system. Even after 
very exhaustive search for such publications, the above 
mentioned only four researches could be found. The proposed 
system is far better than these systems in term of accuracy and 
number of sign dictionary. 



Figure 9: Recall precision and F measure analysis using Radar plot of 
Dataset combined 


Apart from accuracy analysis, Comparative time analysis 
is also performed for PSL-FM. This time analysis is just a way 
of comparing the time efficiency of different alternatives for 
PSL recognition. The proposed methodology does not involve 
any training so training time is zero. SVM is the most costly 
classifier in terms of training time as it needs training time 
once for the training set and once for the support vector 
training time. The recognition of a sign is also not taking 
considerable time. So not only the accuracy, the time 
efficiency of PSL-FM is also quite high if compared to other 
competitive techniques. 


Figure 9 shows the analysis of PSL-FM using recall, 
precision and F measure for each sign of the dataset.The 
Figure lOshows the comparative analysis of different PSL 
recognition systems. The comparison reinforces the fact 


Volume: 01, Issue: 01, November 2016 
ISBN: 978-0-9957075-0-4 


www.ijeacs.com 


28 



SumairaKausaret. al. 


International Journal of Engineering and Applied Computer Science (IJEACS) 



The multi-aspect analysis of PSL-FM has shown that the 
proposed methodology has outperformed many other 
classification systems. The system can be considered as a 
framework that can get tailored for different gesture 
recognition systems. 

V. Conclusion 

The paper presents a signer-independent robust and 
efficient classification method for PSL. The proposed 
classification method can be used as sort of a framework that 
can get tuned for other sign languages by empirical analysis 
for readjustment of fuzzy rules. Parameterization of shape 
signatures as sign descriptor is presented. Main contribution of 
the paper is on classification module, fuzzy classifier is 
proposed. The paper is an effort to contribute in the services 
area for a special group of humanity i.e. Deaf society. This is a 
research area that should have ongoing research. The proposed 
methodology has produced very high accuracy rate for 
recognition of PSL. The approach of mathematical modeling 
can be further extended to the dynamic sign language 
recognition. The dataset for PSL can be extended further for 
future research. 
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